Skip to content

fix(core): report systemic authority refusals and retain runner timeout traces - #1034

Open
Lykhoyda wants to merge 10 commits into
mainfrom
implement-react-native-development-agent-features
Open

Lykhoyda wants to merge 10 commits into
mainfrom
implement-react-native-development-agent-features

Conversation

@Lykhoyda

@Lykhoyda Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Intent

Captain (2026-09-15): "#1034 should be fixed" and "don't worry about the tokens. Finish the 1034". Earlier: address the review feedback on that same PR.

Make #1034 landable on the same branch. Keep the reporting intent of #981 (systemic authority-refusal recognition and reporting). Do not rewrite the authority system. Android QA is still FAILED; iOS already passed at this pair.

A1. In authority-refusal.ts, drop axis from the systemic key so one code+platform is one group across emission shapes (gate-emitted meta.axis, plain Error, failResult with no meta, legacy prose). Axis stays a reported attribute; conflicting or missing axis becomes null on the group. Add the mixed-emission regression.

A2. Recorder writes authorityRefusal: null as an explicit negative verdict on new failure records; reporter already skips present-null and must not re-admit via prose. Persist that explicit null on matching historical-row upserts. Do not backfill every untouched historical row.

B1. Preserve timeout traces: recognize code-less cdp_run_action TIMEOUT (and the producer timedOut surface), add only flow-park and flow-stage diagnostic events around existing park/stage/relaunch/cleanup/origin boundaries. Do not change errors, cleanup, authority, or timeouts.

B2. Change workspace qa-observe-screen-match.yaml launchApp.stopApp true to false only if a new current-session trace proves the cold native launch is the causal failure. That fixture edit is separately routed; do not invent a product redesign if the trace disproves it.

What Changed

  • The experience recorder now decodes authority refusals (SESSION_AUTHORITY_REQUIRED, METRO_ORIGIN_MISMATCH, RUNNER_OWNERSHIP_MISMATCH, HANDOFF_NOT_AUTHORIZED, NON_GIT_MANIFEST_REQUIRED, BUNDLE_HANDSHAKE_UNAVAILABLE) from gate envelopes, plain errors, and legacy prose into dedicated FF_* families, stores an explicit authorityRefusal verdict (or null) plus a code-and-platform systemicKey on each record, and no longer marks a refusal as recovered when a later diagnostic call succeeds.
  • The trend report and experience-trends CLI gain a systemicRefusals section that groups retained refusals per code and platform across tools, merges conflicting axis/cause to null, labels recovery as not verified and current authority state as unknown, and documents that --since only scopes new patterns.
  • Runner diagnostics now recognize code-less cdp_run_action TIMEOUT and maestro_run timedOut failures and emit flow-park / flow-stage lifecycle events around park, stage, relaunch, origin, and cleanup boundaries; stored symptoms and the feedback collector scripts additionally redact UUID-shaped identifiers (redaction rules bumped to v2), with the six refusal families added to the seed failure catalog and a patch changeset.

Risk Assessment

✅ Low: Every required intent criterion (A1 axis-free key with null-on-conflict, A2 explicit null verdict on new and upserted rows without backfill, B1 timeout recognition plus observation-only diagnostics, B2 fixture untouched) is source-verified with behavioral tests, the store is explicitly non-load-bearing, the diagnostics inserts cannot throw or alter control flow, and no wrong-value path was found; the only findings are a removable constant-field trio and a changeset wording nit.

Testing

Built the core package, ran the seven targeted unit test files for the refusal decoder, recorder, systemic trends, trends CLI, relaunch, runner diagnostics, and instrumentation (all passed), then demonstrated the intent end to end by driving the real recorder and the compiled rn-experience-trends CLI: mixed emission shapes of one code+platform collapse into a single systemic group with null axis and the explicit-null verdict is not re-admitted via prose, and a code-less cdp_run_action TIMEOUT retains its flow-park/flow-stage trace bundle. The B2 fixture is untouched in the diff. Transient build outputs and node_modules were removed from the worktree.

Evidence: End-to-end transcript: stored rows, CLI text report, JSON systemic group, timeout bundle events
# A1/A2 — four emission shapes of METRO_ORIGIN_MISMATCH on android, plus one negative

## Stored rows (tool | authorityRefusal verdict | symptom)
- maestro_run  | null | METRO_ORIGIN_MISMATCH: prose only, unrecognised code
- cdp_connect  | {"code":"METRO_ORIGIN_MISMATCH","axis":null,"cause":null} | METRO_ORIGIN_MISMATCH: metro origin differs from bound session
- device_find  | {"code":"METRO_ORIGIN_MISMATCH","axis":null,"cause":null} | origin refused
- rn_session   | {"code":"METRO_ORIGIN_MISMATCH","axis":"M","cause":null} | origin refused
- legacy_tool  | <absent: legacy row> | METRO_ORIGIN_MISMATCH: legacy refusal recorded before the classifier existed

## `rn-experience-trends --since 2026-06-01` (text)
Experience trends (new since 2026-06-01T00:00:00.000Z)
Report generated at 2026-09-15T18:02:16.805Z; pass this value to --since next time.
Systemic, family, and recurring totals cover retained local history, not exact time-window counts.
--since affects only new-pattern selection.

Families by frequency
  FF_METRO_ORIGIN_MISMATCH: 3 occurrence(s), 3 pattern(s)
  UNKNOWN: 2 occurrence(s), 2 pattern(s)

New since previous report
  FF_METRO_ORIGIN_MISMATCH cdp_connect: 1 (a854dcfa91c4)
  FF_METRO_ORIGIN_MISMATCH device_find: 1 (c94c86b077e5)
  FF_METRO_ORIGIN_MISMATCH rn_session: 1 (e33933baccb6)
  UNKNOWN maestro_run: 1 (6e120afcc98f)
  UNKNOWN legacy_tool: 1 (legacy-prose)

Recurring
  none

Systemic authority refusals (retained local history)
  Current authority state: unknown; historical observations do not establish a currently blocked session.
  METRO_ORIGIN_MISMATCH | axis: unknown | cause: unknown | platform: android
    4 occurrence(s) | tools: cdp_connect, device_find, legacy_tool, rn_session | recurring: yes | recovery not verified
    first seen: 2026-09-01T00:00:00.000Z | last seen: 2026-09-15T10:02:00.000Z

## systemicRefusals from --json
[
  {
    "systemicKey": "45f2aa9e986948d4828c771830cc5bc01da066ed7d1cabb94f70327bc7a5d902",
    "classification": "FF_METRO_ORIGIN_MISMATCH",
    "code": "METRO_ORIGIN_MISMATCH",
    "axis": null,
    "cause": null,
    "platform": "android",
    "count": 4,
    "tools": [
      "cdp_connect",
      "device_find",
      "legacy_tool",
      "rn_session"
    ],
    "memberSignatures": [
      "a854dcfa91c4dee6c3d747e3401f0dabdf3b38494f67d971fd1b5a0d929279db",
      "c94c86b077e5a3c8cddf29aa28136f7cf3ff4717de8a25eba2de403ebcf43a61",
      "e33933baccb6c632bd866c8a15d68f6179394a45718650a8526f5fa0b83b4cfb",
      "legacy-prose-row"
    ],
    "firstSeen": "2026-09-01T00:00:00.000Z",
    "lastSeen": "2026-09-15T10:02:00.000Z",
    "recurring": true,
    "recoveryEvidence": "not-verified",
    "currentAuthorityState": "unknown",
    "scope": "retained-local-history",
    "provenance": [
      "legacy-derived",
      "recorded"
    ]
  }
]

A1/A2 assertions passed: 1 group, axis=null, count=4, maestro_run excluded, legacy joined.

# B1 — code-less cdp_run_action TIMEOUT keeps its flow-park / flow-stage trace
retained bundle: runner-diagnostics-authenticated-session-1.json  failureCode=TIMEOUT  actionId=qa-observe-screen-match
   1 flow-park          {"phase":"begin","platform":"android"}
   2 flow-park          {"phase":"released","platform":"android"}
   3 flow-park          {"phase":"committed","platform":"android"}
   4 flow-stage         {"phase":"execute-begin","stage":0}
   5 runner-exec-begin  {"runnerPinVersion":"1.1.24"}

B1 assertions passed: TIMEOUT without a code retained its trace with flow-park begin/released/committed and flow-stage events.
Evidence: rn-experience-trends --json report showing one METRO_ORIGIN_MISMATCH/android group across four emission shapes
{
  "generatedAt": "2026-09-15T18:02:16.858Z",
  "since": "2026-06-01T00:00:00.000Z",
  "families": [
    {
      "classification": "FF_METRO_ORIGIN_MISMATCH",
      "count": 3,
      "patterns": 3
    },
    {
      "classification": "UNKNOWN",
      "count": 2,
      "patterns": 2
    }
  ],
  "newSincePreviousReport": [
    {
      "signature": "a854dcfa91c4dee6c3d747e3401f0dabdf3b38494f67d971fd1b5a0d929279db",
      "classification": "FF_METRO_ORIGIN_MISMATCH",
      "tool": "cdp_connect",
      "count": 1,
      "firstSeen": "2026-09-15T10:01:00.000Z",
      "lastSeen": "2026-09-15T10:01:00.000Z"
    },
    {
      "signature": "c94c86b077e5a3c8cddf29aa28136f7cf3ff4717de8a25eba2de403ebcf43a61",
      "classification": "FF_METRO_ORIGIN_MISMATCH",
      "tool": "device_find",
      "count": 1,
      "firstSeen": "2026-09-15T10:02:00.000Z",
      "lastSeen": "2026-09-15T10:02:00.000Z"
    },
    {
      "signature": "e33933baccb6c632bd866c8a15d68f6179394a45718650a8526f5fa0b83b4cfb",
      "classification": "FF_METRO_ORIGIN_MISMATCH",
      "tool": "rn_session",
      "count": 1,
      "firstSeen": "2026-09-15T10:00:00.000Z",
      "lastSeen": "2026-09-15T10:00:00.000Z"
    },
    {
      "signature": "6e120afcc98fe07497607b2b66c24ea97d918e0e2a6f6d4b9d23b5ca97eca9f9",
      "classification": "UNKNOWN",
      "tool": "maestro_run",
      "count": 1,
      "firstSeen": "2026-09-15T10:03:00.000Z",
      "lastSeen": "2026-09-15T10:03:00.000Z"
    },
    {
      "signature": "legacy-prose-row",
      "classification": "UNKNOWN",
      "tool": "legacy_tool",
      "count": 1,
      "firstSeen": "2026-09-01T00:00:00.000Z",
      "lastSeen": "2026-09-01T00:00:00.000Z"
    }
  ],
  "recurring": [],
  "systemicRefusals": [
    {
      "systemicKey": "45f2aa9e986948d4828c771830cc5bc01da066ed7d1cabb94f70327bc7a5d902",
      "classification": "FF_METRO_ORIGIN_MISMATCH",
      "code": "METRO_ORIGIN_MISMATCH",
      "axis": null,
      "cause": null,
      "platform": "android",
      "count": 4,
      "tools": [
        "cdp_connect",
        "device_find",
        "legacy_tool",
        "rn_session"
      ],
      "memberSignatures": [
        "a854dcfa91c4dee6c3d747e3401f0dabdf3b38494f67d971fd1b5a0d929279db",
        "c94c86b077e5a3c8cddf29aa28136f7cf3ff4717de8a25eba2de403ebcf43a61",
        "e33933baccb6c632bd866c8a15d68f6179394a45718650a8526f5fa0b83b4cfb",
        "legacy-prose-row"
      ],
      "firstSeen": "2026-09-01T00:00:00.000Z",
      "lastSeen": "2026-09-15T10:02:00.000Z",
      "recurring": true,
      "recoveryEvidence": "not-verified",
      "currentAuthorityState": "unknown",
      "scope": "retained-local-history",
      "provenance": [
        "legacy-derived",
        "recorded"
      ]
    }
  ]
}
Evidence: Retained runner-diagnostics bundle for a code-less cdp_run_action TIMEOUT with flow-park/flow-stage events
{
  "schema": "rn-dev-agent/runner-diagnostics/1",
  "candidate": {
    "pluginVersion": "1.0.8",
    "coreVersion": "1.0.8",
    "releaseCommit": null
  },
  "runner": {
    "version": null,
    "provenance": null,
    "payloadShaPrefix": null
  },
  "context": {
    "platform": "android",
    "os": "darwin 25.6.0",
    "runtime": "v26.8.1",
    "sessionId": "authenticated-session",
    "actionId": "qa-observe-screen-match",
    "deviceIdHash": null,
    "bundleId": null,
    "metroPort": null
  },
  "failureCode": "TIMEOUT",
  "events": [
    {
      "sequence": 1,
      "monotonicMs": 0.135,
      "timestamp": "2026-09-15T18:02:16.872Z",
      "type": "flow-park",
      "detail": {
        "phase": "begin",
        "platform": "android"
      }
    },
    {
      "sequence": 2,
      "monotonicMs": 0.196,
      "timestamp": "2026-09-15T18:02:16.872Z",
      "type": "flow-park",
      "detail": {
        "phase": "released",
        "platform": "android"
      }
    },
    {
      "sequence": 3,
      "monotonicMs": 0.206,
      "timestamp": "2026-09-15T18:02:16.872Z",
      "type": "flow-park",
      "detail": {
        "phase": "committed",
        "platform": "android"
      }
    },
    {
      "sequence": 4,
      "monotonicMs": 0.218,
      "timestamp": "2026-09-15T18:02:16.872Z",
      "type": "flow-stage",
      "detail": {
        "phase": "execute-begin",
        "stage": 0
      }
    },
    {
      "sequence": 5,
      "monotonicMs": 0.225,
      "timestamp": "2026-09-15T18:02:16.872Z",
      "type": "runner-exec-begin",
      "detail": {
        "runnerPinVersion": "1.1.24"
      }
    }
  ],
  "truncated": false
}
Evidence: Demo script used to generate the end-to-end evidence
import assert from 'node:assert/strict';
import { spawnSync } from 'node:child_process';
import { appendFileSync, mkdtempSync, readFileSync, readdirSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';

const CORE = process.env.CORE;
const EV = process.env.EV;
const { ExperienceRecorder, EXPERIENCE_STORE_NAME, REDACTION_RULES_VERSION } = await import(`${CORE}/dist/experience/evidence.js`);
const { withRunnerDiagnosticsContext, recordRunnerDiagnostic, snapshotRunnerDiagnostics } = await import(`${CORE}/dist/experience/runner-diagnostics.js`);
const { runFlowParked } = await import(`${CORE}/dist/tools/maestro-run.js`);

const dir = mkdtempSync(join(tmpdir(), 'rn-e2e-systemic-'));
const out = [];
const log = (s = '') => { out.push(s); console.log(s); };
let time = Date.parse('2026-09-15T10:00:00.000Z');
const recorder = new ExperienceRecorder({ directory: dir, coreVersion: '1.0.8', pluginVersion: '1.0.8', now: () => new Date(time), schedule: (w) => w() });
const observe = (input) => { recorder.observe({ params: { platform: 'android' }, latencyMs: 5, ...input }); time += 60_000; };

log('# A1/A2 — four emission shapes of METRO_ORIGIN_MISMATCH on android, plus one negative');
observe({ tool: 'rn_session', status: 'FAIL', result: { ok: false, code: 'METRO_ORIGIN_MISMATCH', error: 'origin refused', meta: { axis: 'M' } } });
observe({ tool: 'cdp_connect', status: 'ERROR', error: 'METRO_ORIGIN_MISMATCH: metro origin differs from bound session' });
observe({ tool: 'device_find', status: 'FAIL', result: { ok: false, code: 'METRO_ORIGIN_MISMATCH', error: 'origin refused' } });
observe({ tool: 'maestro_run', status: 'FAIL', result: { ok: false, code: 'FUTURE_CODE', error: 'METRO_ORIGIN_MISMATCH: prose only, unrecognised code' } });
const legacyRow = {
  signature: 'legacy-prose-row', candidate: { pluginVersion: '0.9.0', coreVersion: '0.9.0' }, environment: { os: 'darwin', node: 'v24' },
  platform: 'android', device: null, runtime: null, phase: 'tool', trigger: 'FAIL reported by legacy_tool', maskingCondition: null,
  symptom: 'METRO_ORIGIN_MISMATCH: legacy refusal recorded before the classifier existed', recovery: null, cleanup: null,
  classification: 'UNKNOWN', evidencePointers: ['event:legacy'], tool: 'legacy_tool', status: 'FAIL',
  normalizedSymptomShape: 'metro_origin_mismatch: legacy refusal', count: 1, recoveryCount: 0,
  firstSeen: '2026-09-01T00:00:00.000Z', lastSeen: '2026-09-01T00:00:00.000Z', lastRecoveredAt: null,
  unknownReasons: { recovery: 'recovery not verified' }, redactionVersion: REDACTION_RULES_VERSION,
};
appendFileSync(join(dir, EXPERIENCE_STORE_NAME), JSON.stringify(legacyRow) + '\n');

log('\n## Stored rows (tool | authorityRefusal verdict | symptom)');
for (const r of recorder.read()) log(`- ${r.tool.padEnd(12)} | ${'authorityRefusal' in r ? JSON.stringify(r.authorityRefusal) : '<absent: legacy row>'} | ${r.symptom}`);

const run = (args) => spawnSync(process.execPath, [`${CORE}/dist/experience-trends.js`, ...args], { cwd: dir, env: { ...process.env, RN_DEV_AGENT_EXPERIENCE_DIR: dir, RN_PROJECT_ROOT: dir }, encoding: 'utf8' });
const text = run(['--since', '2026-06-01T00:00:00.000Z']);
assert.equal(text.status, 0, text.stderr);
log('\n## `rn-experience-trends --since 2026-06-01` (text)');
log(text.stdout.trimEnd());
const json = run(['--since', '2026-06-01T00:00:00.000Z', '--json']);
assert.equal(json.status, 0, json.stderr);
const report = JSON.parse(json.stdout);
writeFileSync(join(EV, 'e2e-trend-report.json'), JSON.stringify(report, null, 2));
log('\n## systemicRefusals from --json');
log(JSON.stringify(report.systemicRefusals, null, 2));

assert.equal(report.systemicRefusals.length, 1, 'one code+platform must be one group');
const g = report.systemicRefusals[0];
assert.equal(g.code, 'METRO_ORIGIN_MISMATCH'); assert.equal(g.platform, 'android');
assert.equal(g.axis, null, 'conflicting/missing axis collapses to null');
assert.equal(g.count, 4, 'gate + plain Error + no-meta + legacy prose');
assert.deepEqual(g.tools, ['cdp_connect', 'device_find', 'legacy_tool', 'rn_session']);
assert.ok(!g.tools.includes('maestro_run'), 'explicit-null verdict must not be re-admitted via prose');
assert.deepEqual(g.provenance, ['legacy-derived', 'recorded']);
assert.doesNotMatch(text.stdout, /recovery verified|session is (currently )?blocked/i);
log('\nA1/A2 assertions passed: 1 group, axis=null, count=4, maestro_run excluded, legacy joined.');

log('\n# B1 — code-less cdp_run_action TIMEOUT keeps its flow-park / flow-stage trace');
let snap;
await withRunnerDiagnosticsContext('cdp_run_action', { platform: 'android', actionId: 'qa-observe-screen-match' }, async () => {
  await runFlowParked(async () => { recordRunnerDiagnostic('flow-stage', { phase: 'execute-begin', stage: 0 }); recordRunnerDiagnostic('runner-exec-begin', { runnerPinVersion: '1.1.24' }); return 'ok'; },
    { platform: 'android', releaseAndroidSlot: async () => ({ released: true }), completeRunnerPark: async () => {}, markCdpStale: () => {} });
  snap = snapshotRunnerDiagnostics();
});
const tdir = mkdtempSync(join(tmpdir(), 'rn-e2e-timeout-'));
new ExperienceRecorder({ directory: tdir, coreVersion: '1.0.8', pluginVersion: '1.0.8', sessionId: 'authenticated-session', schedule: (w) => w() }).observe({
  tool: 'cdp_run_action', params: { platform: 'android', actionId: 'qa-observe-screen-match' }, status: 'FAIL', latencyMs: 151815,
  error: 'Maestro flow timed out', result: { ok: false, error: 'Maestro flow timed out', meta: { failureKind: 'TIMEOUT' } }, runnerDiagnostics: snap,
});
const bundles = readdirSync(tdir).filter((f) => f.startsWith('runner-diagnostics-'));
assert.equal(bundles.length, 1, 'timeout trace bundle retained');
const bundle = JSON.parse(readFileSync(join(tdir, bundles[0]), 'utf8'));
writeFileSync(join(EV, 'e2e-timeout-bundle.json'), JSON.stringify(bundle, null, 2));
log(`retained bundle: ${bundles[0]}  failureCode=${bundle.failureCode}  actionId=${bundle.context.actionId}`);
for (const e of bundle.events) log(`  ${String(e.sequence).padStart(2)} ${e.type.padEnd(18)} ${JSON.stringify(e.detail)}`);
assert.equal(bundle.failureCode, 'TIMEOUT');
assert.deepEqual(bundle.events.map((e) => [e.type, e.detail.phase ?? null]), [['flow-park','begin'],['flow-park','released'],['flow-park','committed'],['flow-stage','execute-begin'],['runner-exec-begin',null]]);
log('\nB1 assertions passed: TIMEOUT without a code retained its trace with flow-park begin/released/committed and flow-stage events.');

writeFileSync(join(EV, 'e2e-systemic-refusal-transcript.md'), out.join('\n') + '\n');
rmSync(dir, { recursive: true, force: true }); rmSync(tdir, { recursive: true, force: true });
Evidence: CLI text report excerpt (systemic section)
Systemic authority refusals (retained local history)
Current authority state: unknown; historical observations do not establish a currently blocked session.
METRO_ORIGIN_MISMATCH | axis: unknown | cause: unknown | platform: android
4 occurrence(s) | tools: cdp_connect, device_find, legacy_tool, rn_session | recurring: yes | recovery not verified
first seen: 2026-09-01T00:00:00.000Z | last seen: 2026-09-15T10:02:00.000Z
Evidence: Targeted unit test transcript (122 pass, 0 fail)
✔ SESSION_AUTHORITY_REQUIRED: direct, first-content, and anchored thrown observations (0.987375ms)
✔ METRO_ORIGIN_MISMATCH: direct, first-content, and anchored thrown observations (0.098416ms)
✔ RUNNER_OWNERSHIP_MISMATCH: direct, first-content, and anchored thrown observations (0.093584ms)
✔ HANDOFF_NOT_AUTHORIZED: direct, first-content, and anchored thrown observations (0.071833ms)
✔ NON_GIT_MANIFEST_REQUIRED: direct, first-content, and anchored thrown observations (0.072167ms)
✔ BUNDLE_HANDSHAKE_UNAVAILABLE: direct, first-content, and anchored thrown observations (0.0595ms)
✔ structured code wins over other envelopes, thrown prefixes, and conflicting prose (0.137083ms)
✔ only fixed envelope fields and the first content item are eligible (0.11225ms)
✔ JSON parsing is bounded by UTF-8 bytes including the exact 16 KiB boundary (0.172459ms)
✔ axes are observed, never inferred, and no code admits a cause yet (0.482667ms)
✔ systemic identity is a versioned tuple with explicit unknown slots (0.412916ms)
✔ common metadata stays unknown after missing or conflicting observations (0.09175ms)
✔ a meaningful failure writes exactly one fully sanitized structured record (4.865791ms)
✔ a bare app display name and slug from app.json are redacted (1.757625ms)
✔ a malformed app.json fails closed instead of shipping raw symptoms (1.658167ms)
✔ a payload-heavy symptom is bounded before it reaches the store (0.792083ms)
✔ a corrupt store line is dropped instead of disabling recording forever (9.595083ms)
✔ a record stored under older redaction rules is re-sanitized before rewrite (1.614834ms)
✔ a later event fills a previously unknown device without losing the record (1.157542ms)
✔ an ERROR occurrence is not downgraded by a later FAIL with the same shape (1.067542ms)
✔ a thrown-tool ERROR writes one meaningful record (0.509417ms)
✔ redaction failures fail closed with a placeholder, never raw content (0.227833ms)
✔ classification only emits ids that exist in real seed-experience YAML (14.7485ms)
✔ FAIL then immediate PASS on the same tool updates the failure as a recovery (1.887167ms)
✔ ordinary PASS and a non-immediate PASS are not recorded as recoveries (0.720625ms)
✔ dedupe signature is stable across runs, paths, ports, timestamps, and ids (1.103083ms)
✔ a throwing or arbitrarily slow recorder is outside the tool path (1.119708ms)
✔ pruning is deterministic by age, recency, then signature (0.090875ms)
✔ trend report is read-only and exposes frequency, new, and recurring patterns (3.616209ms)
✔ SESSION_AUTHORITY_REQUIRED persists exact classification through all observer failure shapes (2.253166ms)
✔ METRO_ORIGIN_MISMATCH persists exact classification through all observer failure shapes (1.373ms)
✔ RUNNER_OWNERSHIP_MISMATCH persists exact classification through all observer failure shapes (1.402459ms)
✔ HANDOFF_NOT_AUTHORIZED persists exact classification through all observer failure shapes (1.810541ms)
✔ NON_GIT_MANIFEST_REQUIRED persists exact classification through all observer failure shapes (1.362125ms)
✔ BUNDLE_HANDSHAKE_UNAVAILABLE persists exact classification through all observer failure shapes (1.283167ms)
✔ unknown structured codes retain text classification without systemic membership (0.478167ms)
✔ oversized envelopes persist an explicit negative verdict and stay excluded (0.745041ms)
✔ repeat negatives keep explicit null on a matching historical keyless row (0.902416ms)
✔ authority observations never become recovery candidates in any adjacency sequence (4.312ms)
✔ same-signature refusal merges preserve counts, evidence, status, and common metadata (6.393ms)
✔ missing historical refusal facts do not backfill metadata or erase recovery counters (1.77575ms)
✔ separate tools share one systemic key for the same code and platform across axes (1.607875ms)
✔ refusal metadata is discarded and platform sanitized before systemic hashing (1.130625ms)
✔ refusal symptoms drop simulator UDIDs while systemic grouping is unchanged (1885.089042ms)
✔ refusal decoding is deferred and decoder failures clear earlier recovery candidates (1.292583ms)
✔ five observations aggregate from three merged tool patterns without changing tool rows (16.221417ms)
✔ five observations aggregate from five one-count tools without changing tool rows (4.174166ms)
✔ five observations aggregate from five one-count patterns across three tools without changing tool rows (3.004625ms)
✔ all recognized codes remain separate even under a shared stored classification (0.264417ms)
✔ conflicting axes collapse to one group whose displayed axis stays unknown (0.250584ms)
✔ a shared axis stays known when every member of the group agrees (0.123667ms)
✔ mixed emission shapes for one code and platform form a single group (0.429834ms)
✔ explicit null recorder verdict is not re-admitted through matching prose (0.084291ms)
✔ known platforms, unknown platform, and the literal unknown string do not join (0.209709ms)
✔ persisted axis and cause values are revalidated with an explicit unknown cause key slot (1.35675ms)
✔ malformed extensions cannot fall back to legacy symptoms, family, or a stored key (0.245625ms)
✔ legacy SESSION_AUTHORITY_REQUIRED joins through direct JSON, first MCP content, and anchored prefixes (0.409417ms)
✔ legacy METRO_ORIGIN_MISMATCH joins through direct JSON, first MCP content, and anchored prefixes (0.219792ms)
✔ legacy RUNNER_OWNERSHIP_MISMATCH joins through direct JSON, first MCP content, and anchored prefixes (0.208ms)
✔ legacy HANDOFF_NOT_AUTHORIZED joins through direct JSON, first MCP content, and anchored prefixes (0.18775ms)
✔ legacy NON_GIT_MANIFEST_REQUIRED joins through direct JSON, first MCP content, and anchored prefixes (0.14875ms)
✔ legacy BUNDLE_HANDSHAKE_UNAVAILABLE joins through direct JSON, first MCP content, and anchored prefixes (0.162166ms)
✔ legacy membership rejects ambiguous, incomplete, nested, and conflicting evidence (1.102333ms)
✔ legacy JSON and prefixes are bounded by UTF-8 bytes before parsing (0.183083ms)
✔ legacy precedence admits only observed allowlisted metadata and the record platform (0.216333ms)
✔ mixed provenance aggregates once without promoting historical recoveries or changing old projections (0.456542ms)
✔ tampered, missing, and shared stored keys cannot split or merge validated groups (0.091958ms)
✔ shuffled inputs and count ties produce deterministic rows and sorted member unions (0.273875ms)
✔ first and last observations are chronological even with timestamp offsets (0.082208ms)
✔ empty history has all four empty collections (0.048042ms)
✔ a future since changes only new-pattern selection, not retained occurrence totals (0.382833ms)
✔ retention-pruned totals include only retained records and reporting does not prune again (0.182625ms)
✔ compiled CLI JSON exposes systemic evidence alongside unchanged tool rows without writing (71.721417ms)
✔ core and both shipped CLIs agree on mixed evidence without rewriting history (637.988334ms)
✔ core and both shipped CLIs agree on legacy-only evidence without rewriting history (625.1425ms)
✔ compiled CLI text prints occurrences, contributors, observations, unknowns, and evidence limits (59.954166ms)
✔ compiled CLI future since leaves retained systemic, family, and recurring totals unchanged (177.792292ms)
✔ compiled CLI reads a missing store as empty without creating it (91.108417ms)
✔ compiled CLI usage explains retained scope and since selection without reading or writing (93.152291ms)
✔ GH#708: a mid-flow relaunch that re-registers late still runs the rest of the flow (1.723916ms)
✔ GH#708: the deferred re-prove never relaunches a second time (0.249917ms)
✔ GH#708: a genuine authority mismatch still fails the run (0.487209ms)
✔ GH#708: a relaunch failure with no re-prove authority still aborts (0.320917ms)
✔ failed execute and failed relaunch-plus-cleanup leave different traces without changing errors (1.588708ms)
✔ stopApp false still records managed relaunch and origin proof (0.276458ms)
✔ GH#708: a revoked session claim aborts immediately instead of deferring (0.198167ms)
✔ GH#993 D2.b: an origin claim failing after the flow relaunch is attributed to the flow (0.362084ms)
✔ GH#993: a warm launchApp {stopApp: false} is not a relaunch and is not blamed (0.1805ms)
✔ GH#993: a proven foreign-Metro mismatch keeps its own cause and remedy (0.227208ms)
✔ GH#993: an origin claim failing with no preceding flow relaunch is not attributed (0.139208ms)
✔ GH#993 D2.e: a relaunch that itself raises METRO_ORIGIN_MISMATCH keeps GH#708 abort semantics (0.174625ms)
✔ GH#993: a plain relaunch failure is still deferred (GH#708) and stays unattributed (0.140834ms)
✔ GH#993: a claim that passes after the relaunch clears the attribution for a later segment (0.188167ms)
✔ GH#993: a relaunch in an earlier segment with no passing claim since is still blamed (0.166709ms)
✔ GH#993: a trailing relaunch whose completion claim fails is attributed to the flow (0.145333ms)
✔ GH#993: a completion claim failing with no relaunch since the last passing claim is not attributed (0.117083ms)
✔ GH#708: maestro_run reports a passing relaunch flow as passing (14.339375ms)
✔ GH#993: the deferred iOS completion is not blamed on a relaunch a later claim already proved (3.991542ms)
✔ GH#993: the native leg claim failing right after the relaunch is blamed on it (2.928042ms)
✔ GH#993: a raw maestro_run relaunch failure keeps its original nextAction and no dev-client advice (2.750708ms)
✔ instrumentation fans out the same event to every observer (2.179208ms)
✔ a throwing observer cannot block another observer or change the tool result (0.635458ms)
✔ detaching an observer removes only its own subscription (0.340833ms)
✔ authority outcomes remain identical with deferred and blocked experience persistence (8.885833ms)
✔ runner diagnostics use a stable salted device hash and redact external bundle IDs (8.858958ms)
✔ runner diagnostics preserve valid action identities before feedback hashing (2962.855584ms)
✔ runner diagnostics retain terminal lifecycle events after the event cap (1.523209ms)
✔ flow-park trace records begin, released, and committed on success (0.266792ms)
✔ flow-park trace records begin only when the release rejects (0.346708ms)
✔ flow-park trace records released but not committed when the commit rejects (0.173667ms)
✔ runner diagnostics retain the owned workspace test-app bundle ID only (1.501833ms)
✔ runner diagnostics retain five bounded bundles and export newest without overwrite (21872.415333ms)
✔ runner diagnostics use numeric sequence order when mtimes tie (1666.764417ms)
✔ feedback upgrades legacy exact-session diagnostics with a private stable salt (2831.387709ms)
✔ feedback rejects legacy diagnostics with a redacted action identity (1524.193333ms)
✔ concurrent feedback atomically publishes one complete legacy salt (7757.19675ms)
✔ feedback provisions and validates salt when exact-session diagnostics omit action ID (2963.440875ms)
✔ runner diagnostics bound metadata before enforcing the absolute byte cap (7.926042ms)
✔ diagnostics exports select only the exact authenticated session (1535.08525ms)
✔ code-less cdp_run_action TIMEOUT retains the existing sanitized trace (2.569625ms)
✔ direct maestro_run producer timeout retains the authenticated session bundle (1.439708ms)
✔ non-timeout, unrelated-tool, foreign-session, and malformed envelopes do not retain traces (12.784625ms)
ℹ tests 122
ℹ suites 0
ℹ pass 122
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ todo 0
ℹ duration_ms 45464.016

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 2 issues (1 warning, 1 info)
  • ⚠️ packages/rn-dev-agent-core/src/experience/trends.ts:48 - Simplification: SystemicRefusalTrend carries three literal-typed constant fields on every row (recoveryEvidence: 'not-verified', currentAuthorityState: 'unknown', scope: 'retained-local-history'). They can only ever hold one value, so they convey no per-row information; the intent's requirement that groups never claim recovery or a currently blocked session is already satisfied by the absence of any recovery/current-state field plus the README and CLI disclaimer lines. No intent requirement (A1/A2/B1/B2 or the bug: repeated authority refusals never surface as one systemic issue #981 reporting intent) needs these fields. Recommended remedy: remove the three fields from the interface, the group constructor, and any test fixtures that assert them. This changes the JSON report shape, so it is the author's call.
  • ℹ️ .changeset/systemic-authority-refusals.md:6 - The changeset body is two sentences; the repo owner's stated convention for changesets monorepos is a single-sentence summary because the text ships to the published CHANGELOG. Collapse to one sentence, e.g. join the UUID-redaction clause with 'and' or fold it into the first sentence.
✅ **Test** - passed

✅ No issues found.

  • yarn install --immutable and yarn workspace rn-dev-agent-core build (tsc) so dist-importing tests could run
  • node --test --test-concurrency=2 test/unit/experience-authority-refusal.test.ts test/unit/experience-evidence.test.ts test/unit/experience-systemic-trends.test.ts test/unit/experience-trends-cli.test.ts test/unit/gh-708-mid-flow-relaunch.test.ts test/unit/runner-diagnostics.test.ts test/unit/instrumentation.test.ts (122 pass, 0 fail)
  • End-to-end demo node e2e-systemic-refusal-demo.mjs (evidence dir): seeded a temp store via ExperienceRecorder with gate-emitted/plain-Error/no-meta/legacy-prose refusals plus an unrecognised-code prose row, ran dist/experience-trends.js --since 2026-06-01 in text and --json, asserted one group, count 4, axis null, maestro_run excluded, provenance legacy-derived+recorded, no recovery or blocked-session claim
  • End-to-end demo B1: ran runFlowParked under withRunnerDiagnosticsContext(&#39;cdp_run_action&#39;), recorded a code-less TIMEOUT via ExperienceRecorder, asserted the retained bundle has failureCode TIMEOUT and flow-park begin/released/committed plus flow-stage execute-begin events
  • Verified the diff does not modify qa-observe-screen-match.yaml (B2 conditional not triggered)
  • Removed generated dist, .yarn/install-state.gz, and node_modules from the worktree after testing
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@Lykhoyda
Lykhoyda marked this pull request as ready for review September 14, 2026 08:28
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-15T18:10:54.264519Z 8ae939d New commits
🔒 Security Review Completed 2026-09-14T08:40:41.546366Z a8a10ed Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor
cursor Bot force-pushed the implement-react-native-development-agent-features branch from a8a10ed to 3956fff Compare September 14, 2026 08:37
@Lykhoyda

Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

FAILED — acceptance incomplete at d4d7de43667e9c00f53d25a608ecdf77c93bc581. The reporting checks exercised with genuine product-emitted refusals passed. The required live iOS journey did not establish usable UI/navigation, and the decoder boundary scenarios below remain NOT RUN. This is evidence for the captain, not a merge gate. A changed or rebased head needs fresh verification.

The brief's earlier conflict/no-CI/draft caveats were checked against GitHub: this head is OPEN, not draft, MERGEABLE, and CI exists. The description still records 8 unresolved earlier local test failures: 7 matching its comparison checkpoint and 1 intermittent hard-reset shutdown proof failure reproduced on both revisions. The checkpoint already contains implementation Phases 1 and 2; it is not pristine main. Those local results were not rerun or relabeled here.

Final GitHub recheck (2026-09-14 10:01 UTC): head unchanged; OPEN, draft=false, MERGEABLE. Check summary: 1 NEUTRAL, 24 SUCCESS.

Observed on the exact candidate, one isolated iOS session:

  • BUNDLE_HANDSHAKE_UNAVAILABLE from cdp_dev_settings(hideDevMenu), cdp_component_tree, and cdp_navigation_state became one systemic group: 3 occurrences, 3 tools, 3 signatures, exact dates, axis B, unknown cause/platform, and recorded provenance. The unknown platform reflects the event payload, despite the iOS session.
  • RUNNER_OWNERSHIP_MISMATCH from device_screenshot remained a separate group, axis R/platform iOS. No refusal envelopes or store records were fabricated or injected.
  • Successful status and inventory calls between refusals left all recognized records at zero recovery count, with recovery unverified and current authority state unknown.
  • A real outside-family code, APP_INSTALL_IDENTITY_CHANGED, stayed UNKNOWN and outside systemic aggregation.
  • Both shipped host CLIs agree in JSON and text. Fixed --since; only generated-at timestamps normalized; both diff -u results empty. JSON includes signatures/provenance. The systemic text section deliberately omits those fields; that is not a defect.
  • Reporting preserved the live store's 8,577 bytes, mtime 1789379682337610823 ns, SHA-256 0c50044cfbf12858ebb38875b786dfef4bcbf27d6ad24b37d307c05abec6a860, and directory contents. A preserved copy of genuine prior history also stayed unchanged and produced 12 legacy-derived groups / 942 retained occurrences; both hosts agreed and member counts/tools/dates matched.

Privacy: CLEAN for the fresh patterns.jsonl inspected. All seven records use redaction version 2. No UUID, known device identifier, known hostname, or local/home path was persisted there. A Metro refusal carrying a real simulator identifier did not occur, so the specific UDID-scrubbing case remains NOT RUN; the clean scan does not prove that missing case.

NOT RUN: fresh MCP cases for SESSION_AUTHORITY_REQUIRED, METRO_ORIGIN_MISMATCH, HANDOFF_NOT_AUTHORIZED, and NON_GIT_MANIFEST_REQUIRED; structured-code/prose disagreement; conflicting later MCP content items; the 16 KiB boundary; Android. The actual structured envelopes contained a single item of 225–377 bytes. Missing cases were not replaced with synthetic tests or inferred from source.

Why live acceptance stopped: first build hit the known simulator install race; the single prescribed stop-Metro/build retry installed and launched the Test App. Hermes then exposed one matching target, but the validator's final process probe returned APP_NOT_RUNNING. Its preserved output eventually contained the Test App entry after about 7.14 seconds, exceeding the 5-second probe limit. This supports a timed-out probe, not an established app crash. Workspace dev/validator/lib/probe.sh:131–133 maps probe errors to an empty process list and then to that misleading code; distinguishing probe unavailability from app absence is actionable follow-up. No third build or alternate native control path was used.

Workspace/Test App base: 0715df24e4b18f027d21158c26176785a7399533, detached. The supplied thin stdio client relayed public candidate MCP calls, with JSON built using jq. simctl output was captured on the boot volume first. The validator's menu-dismissal step was delegated to the required public hideDevMenu request, which refused. The diagnostic image below renders captured results; no app screenshot or navigation proof is claimed.

Cleanup proven: owned runner closed, Metro stopped, integration restored, session released, transport stopped, and final validator cleanup reports empty authority plus absent owned simulator/listener/integration residue. Foreign devices were untouched. The subsystem-removal proposal is outside this verdict.

Diagnostic summary of real refusal and reporting evidence

@Lykhoyda

Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

PASS — tested refusal recognition/reporting at d4d7de43667e9c00f53d25a608ecdf77c93bc581. Four priority codes were emitted by the real candidate on owned iOS/Android targets. This is scoped QA evidence, not a merge gate; any changed or rebased head needs fresh verification.

GitHub currently reports OPEN, not draft, MERGEABLE, with 24 successful checks and one neutral check. The earlier conflict/no-CI caveats no longer match that state. The description’s eight earlier local failures remain documented: seven matching its comparison checkpoint and one flaky hard-reset shutdown proof. That checkpoint includes implementation Phases 1 and 2; these failures were not rerun or relabeled here.

The repaired workspace base was 8c0187e4622694708c397835fc3125dea8ec43cf. The candidate remained clean. A thin stdio client relayed public MCP tools only; no refusal envelopes or experience records were injected.

  • iOS: the validator reached readiness after its one prescribed install-race retry. Native UI, route/store reads and the saved action passed through to the QA result screen. Replay transport was cdp-js with per-step readback.
  • Android: the pinned maestro-runner 1.1.24 action failed at app-root after a first-run tutorial was observed. Its parked runner then produced real screenshot/snapshot refusals. Subsequent pinning timed out with no target; the final screenshot is the dev-client picker, not successful app navigation.
Code Live evidence
BUNDLE_HANDSHAKE_UNAVAILABLE Before pinning and after Android replay/relaunch; multiple CDP/session tools
RUNNER_OWNERSHIP_MISMATCH Android screenshot and snapshot after runner parking, with successful diagnostics between
METRO_ORIGIN_MISMATCH Owned iOS app terminated through public MCP while managed Metro remained; origin gate refused the missing attachment
SESSION_AUTHORITY_REQUIRED Real refusal of integration restoration while Android managed Metro was active
HANDOFF_NOT_AUTHORIZED NOT RUN: no genuine donor/recipient handoff; no fake capability introduced
NON_GIT_MANIFEST_REQUIRED NOT RUN: the prescribed Test App is a Git checkout

The requested pre-bind SESSION_AUTHORITY_REQUIRED variant was not provoked: source auto-binding and earlier install/device gates emitted other codes. The foreign-Metro/UDID variant was also NOT RUN; no second Metro or foreign lane was used.

Fresh reporting checks passed: 17 recognized occurrences / six complete-key groups. The Android runner group spans two tools; the main bundle group spans four. Every key, count, tool set, member signature, date and recorded provenance matched the store. Diagnostics did not record recovery. Both shipped host CLIs agreed in JSON and text after normalizing generated-at timestamps only. JSON carries signatures/provenance; their omission from the systemic text section is intentional.

Reporting left 24,193 bytes, mtime 1789384740834568538 ns, hash and directory contents unchanged. The earlier exact-head legacy result is carried forward as instructed: 12 legacy-derived groups / 942 occurrences, without rewriting history. Structured-code/prose precedence, first-content-item selection and the 16 KiB UTF-8 boundaries are unit-verified by existing coverage; no synthetic acceptance harness was added. Real out-of-set codes remained UNKNOWN.

Privacy: no UUID, known device identifier/serial, hostname or local/home path was found in any field of the 19 fresh stored records. All use redaction version 2. The actual Metro refusal contained no UDID, so this is not live proof of the unobserved UDID-bearing variant.

Reverse cleanup passed with cleanupProven=true: owned devices/Metro/transport removed, integrations restored, foreign inventories unchanged. The Android tutorial/launch-to-picker failure merits separate follow-up and is not attributed to this reporting diff. The subsystem-removal proposal was not evaluated.

Evidence: issue-981-pr1034-qa-a8a1/qa2-d4d7de43/, including the standalone report, original sanitized MCP results, both CLI formats and diffs, store-integrity/privacy checks, and cleanup proof.

iOS result after the passing saved action Android diagnostic picker after the failed native replay; not successful app navigation

@Lykhoyda

Lykhoyda commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

FAILED — independent exact-head QA of #1034 at d4d7de43667e9c00f53d25a608ecdf77c93bc581 against workspace 7a63707af91b8bec20cab76f902a1553985251e6.

A failed native Android journey cannot become a full PASS. This is evidence for the captain, not a merge gate. No approval or merge decision.

The Android first native surface after a fresh owned build/pin/attachOnly was the Expo development-server picker, not the app. Pin exhausted 120s with 0 Hermes targets. The APK does contain EXDevMenuIsOnboardingFinished=true and EXDevMenuShowsAtLaunch=false; there was no Expo tutorial. One public device_deeplink of the same managed URL then showed app onboarding with app-root and a successful pin, after which APP_INSTALL_IDENTITY_CHANGED blocked hideDevMenu and the unchanged qa-observe-screen-match replay (the YAML never ran). Repeat was not reachable. Product source, stopApp, routes, and assertions were not patched.

iOS at this same pair passed: usable Welcome baseline, hide surface:app, strict replay and repeat both passed:true with transport cdp-js / proofDomain=react-tree, route QaAcceptanceResult.

Reporting recheck on this run’s private store: four real codes (BUNDLE_HANDSHAKE_UNAVAILABLE, RUNNER_OWNERSHIP_MISMATCH, SESSION_AUTHORITY_REQUIRED, HANDOFF_NOT_AUTHORIZED). accept_handoff without credentials produced HANDOFF_NOT_AUTHORIZED: session is not a capability-bound recovery contender. SESSION is the active Metro+runner restore guard, not the pre-bind variant (NOT RUN). NON_GIT_MANIFEST_REQUIRED NOT RUN. METRO_ORIGIN_MISMATCH NOT RUN here. Live UDID-bearing Metro variant NOT RUN. Both shipped CLIs agree after normalizing generated timestamps; JSON has signatures/provenance; text omits them. Fresh store and legacy corpus hashes unchanged by reporting. 14 fresh rows redactionVersion: 2, privacy-clean for observed inputs.

Cleanup: owned runner/Metro/transport closed, integration restored, owned simulator and emulator removed. Foreign inventory unchanged (two booted iOS simulators; physical USB phone untouched).

Android first native surface: Expo development-server picker Android after public managed deeplink: app onboarding with app-root iOS usable baseline: app Welcome onboarding iOS result after strict saved-action replay

@Lykhoyda

Copy link
Copy Markdown
Owner Author

FAILED — QA input-correction experiment for #1034 at d4d7de43667e9c00f53d25a608ecdf77c93bc581 against workspace 7a63707af91b8bec20cab76f902a1553985251e6.

Binding the existing Expo Dev Client URL before the owned Android build did not produce a visible app result from the original saved action. This is evidence for the captain, not a merge gate. No approval or merge decision. No product source was patched.

What changed vs prior Android runs. Previous exact-head QA bound the device without a URL, then built, then pinned (0 Hermes targets / 120s; first native surface was the Expo picker). This run called rn_session bind_device with devClientUrl=exp+rn-dev-agent-test://expo-development-client/?url=http%3A%2F%2F10.0.2.2%3A8095 (the reserved session Metro port) before pnpm run android. The signed receipt carried that same devClientUrl. The adapter opened that URL on the owned emulator QA_PR1034_bindfirst. installIdentity stayed verified.

What still failed. pin_dev_client still returned BUNDLE_HANDSHAKE_UNAVAILABLE / CDP_TARGET_AUTHORITY_MISMATCH: expected one target on the exact device, found 0, Metro 8095, 120s deadline. Unchanged qa-observe-screen-match (stopApp: true, assertVisible app-root; YAML sha256 2f3521de6b5a0e6007d82d17fc1bf33d28109df563ae084ece4d8707b78132a1) never executed: cdp_run_action refused RUNNER_OWNERSHIP_MISMATCH after device_snapshot open could not start the Android runner (RN_ANDROID_RUNNER_DOWN, no /health within 30s). Repeat was not reachable. No native screenshot this run.

Picker-first and APP_INSTALL_IDENTITY_CHANGED are the known separate Android launch issue from prior runs. This experiment does not use them as verdict reasoning. APP_INSTALL_IDENTITY_CHANGED did not appear here.

iOS, CLI/reporting, and Desktop were NOT RUN (preserved / out of scope). Foreign iOS simulators and the attached USB phone were not touched. Owned emulator and Metro were removed; package scripts restored to expo run:ios / expo run:android.

The bind-first input correction did not avoid the Android saved-action wall. No product patch is justified from this run.

if (!isAuthorityRefusalCode(code)) return null;
return {
code,
axis: AUTHORITY_AXES.find((candidate) => candidate === axis) ?? null,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Product-wrong: axis in the systemic key splits one refusal into several rows by emission path.

This reads axis from meta.axis and authorityRefusalSystemicKey hashes it into the group key. But the product does not emit axis consistently for the same code:

  • SessionAuthorityError through authorityGate.wrapauthorityFailurefailResult(msg, code, authorityErrorMeta(err))meta.axis set from registry.ts errorAxes (BUNDLE_HANDSHAKE_UNAVAILABLE → 'B', RUNNER_OWNERSHIP_MISMATCH → 'R', …).
  • Plain Error('BUNDLE_HANDSHAKE_UNAVAILABLE: …') (index.ts:910, :1395, dev-client-authority.ts:90, :286) → authorityFailure fallback → failResult(message, code) with no metaaxis: null.
  • failResult(msg, 'RUNNER_OWNERSHIP_MISMATCH') at agent-device-wrapper.ts:2040 / tools/device-session.ts:487 → no meta → axis: null.
  • Every pre-PR row: the stored symptom is prose CODE: message, so decodeLegacyAuthorityRefusal gives axis: null. Legacy evidence can never join a gate-emitted group.
  • HANDOFF_NOT_AUTHORIZED is not in errorAxes at all, so even the gate emits it without an axis.

Reproduced with the shipped CLI at this head (claude-plugin/rn-dev-agent-core/dist/experience-trends.js --json) on three BUNDLE_HANDSHAKE_UNAVAILABLE/android rows (gate-emitted axis:'B' count 3; no-meta axis:null count 2; legacy prose count 4):

[
  { "code": "BUNDLE_HANDSHAKE_UNAVAILABLE", "axis": null, "platform": "android", "count": 6, "tools": ["cdp_status","device_snapshot"], "provenance": ["legacy-derived","recorded"] },
  { "code": "BUNDLE_HANDSHAKE_UNAVAILABLE", "axis": "B",  "platform": "android", "count": 3, "tools": ["cdp_run_action"], "provenance": ["recorded"] }
]

That is #981's "N unrelated rows instead of one systemic pattern" again, at a coarser grain. experience-systemic-trends.test.ts:462-473 pins the split as intended (unknown-axis gets its own group), so the tests encode the gap rather than catch it; the "legacy joins recorded" fixture there uses a JSON symptom the pre-PR recorder never wrote (legacy symptoms were extractSymptomparsed.error, i.e. prose).

Suggested fix: for the six supported codes the axis is a fixed function of the code (authority-gate.ts axisErrors / registry.ts errorAxes). Derive it from a code→axis table here and treat a supplied meta.axis that disagrees as conflicting metadata (refuse/unknown, per this PR's own precedence rule) — or drop axis from the key and keep it as a reported attribute. Add a regression: same code + platform, one row with meta.axis, one without, one legacy prose row → exactly one systemic group. The changeset's "including unambiguous legacy evidence" only becomes true after this.

Comment on lines +109 to +113
const provenance = Object.hasOwn(record, 'authorityRefusal') ? 'recorded' : 'legacy-derived';
const facts =
provenance === 'recorded'
? recordedRefusalFacts(record.authorityRefusal)
: decodeLegacyAuthorityRefusal(record.symptom);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correctness: the reporter re-admits via prose what the recorder declined via structure.

Every record without an authorityRefusal key is treated as legacy and decoded from record.symptom. But post-PR the recorder also writes key-less records when it evaluated the event and concluded "not a refusal":

  • structured code outside the six while the error prose starts with a refusal code → recorder: structured wins, no facts; reporter: prose wins, refusal;
  • envelope over MAX_AUTHORITY_ENVELOPE_BYTES (16 KiB) → decodeAuthorityRefusalPayload returns null, but the symptom is still CODE: … → reporter admits it as legacy-derived with axis: null (feeding the axis split noted in authority-refusal.ts).

So "Structured codes take precedence over prose, including unknown codes" holds for the recorder but not for the report. Suggested fix: make the recorder's verdict explicit — write authorityRefusal: null on every new failure record in buildFailureRecord — and here skip records where the key is present and null; legacy-decode only when the key is genuinely absent. recordedRefusalFacts' malformed-extension handling is fine as is.

Comment on lines +17 to +24
const REFUSAL_CAUSES = {
SESSION_AUTHORITY_REQUIRED: [],
METRO_ORIGIN_MISMATCH: [],
RUNNER_OWNERSHIP_MISMATCH: [],
HANDOFF_NOT_AUTHORIZED: [],
NON_GIT_MANIFEST_REQUIRED: [],
BUNDLE_HANDSHAKE_UNAVAILABLE: [],
} as const satisfies Record<AuthorityRefusalCode, readonly string[]>;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low: cause is a dead dimension right now. Every vocabulary here is empty, and authorityErrorMeta (registry.ts:375) never emits cause even though SessionAuthorityError.details.cause exists (startup-cleanup.ts:205 sets managed-metro-stop-proof-missing). Result: cause is always null in the key and the text report always prints cause: unknown. Not blocking, but either wire details.cause into authorityErrorMeta with a first vocabulary entry, or drop the column from the text output until one exists so the report does not advertise a dimension nothing populates.

// Bump whenever a redaction rule changes: stored records stamped with an older
// version are re-sanitized under the current rules before the next rewrite.
export const REDACTION_RULES_VERSION = 1;
export const REDACTION_RULES_VERSION = 2;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Low / non-blocking, but two silent side effects of the version bump plus the new [ID_REDACTED] rule (line 81) are worth a line in the changeset:

  1. Legacy rows carry evidencePointers: ["event:<uuid-with-dashes>", …]. Re-sanitization turns each into event:[ID_REDACTED], and the next boundedPointers Set-dedupe collapses up to three pointers into one. Acceptable (they were opaque), but nothing announces it.
  2. normalizeSymptomShape runs on the already-sanitized symptom. Pre-PR a UDID normalized to <id>; now it is the literal [id_redacted] (11 chars, no digit, so the <id> rule does not match). A UUID-bearing failure that was recurring before the upgrade opens a fresh signature once after it, and the old row ages out over 14 days. One-time discontinuity, no action needed beyond awareness.

@cursor

cursor Bot commented Sep 15, 2026

Copy link
Copy Markdown

Code review summary (head d4d7de43)

Scope: source diff under packages/rn-dev-agent-core/src/experience/*, experience-trends.ts, seed YAML, changeset. Generated host dist/ bundles not hand-reviewed (CI check-dist-fresh covers them). This is a code review, not a device QA rerun; the existing QA outcomes on this PR are not re-evaluated here. No approve / request-changes state is set.

Overall. The implementation is careful and matches its stated boundaries: bounded envelope decoding on content[0].text only, structured-before-prose precedence in the recorder, refusals excluded from immediate-recovery linking (with previousFailure reset before diagnostics so a throw cannot leave a stale link), only code/axis/cause persisted from the envelope (no holder/expected/observed/nextAction in the store), order-independent aggregation that recomputes the key instead of trusting the stored one, and good test coverage. Seed YAML entries and EXPERIENCE_FAMILY_IDS are consistent.

What I think is wrong before this should land (details inline):

  1. axis in the systemic key splits one refusal into multiple rows by emission path (authority-refusal.ts:101). Gate-emitted SessionAuthorityError carries meta.axis; plain Error('CODE: …') throws, the two failResult(msg, 'RUNNER_OWNERSHIP_MISMATCH') sites, and every pre-PR legacy row do not. Reproduced with the shipped CLI at this head: three BUNDLE_HANDSHAKE_UNAVAILABLE/android rows produce two systemic groups. That is bug: repeated authority refusals never surface as one systemic issue #981's core symptom re-created at a coarser grain, and it also makes the changeset's "including unambiguous legacy evidence" untrue in practice, since legacy prose evidence can never join a gate-emitted group. Fix: derive axis from a fixed code→axis table (the six codes already have one in authority-gate.ts/registry.ts) or take axis out of the key; add a mixed-emission regression.
  2. Reporter re-admits via prose what the recorder declined via structure (trends.ts:109-113). Post-PR records without an authorityRefusal key are not only legacy rows; they are also events the recorder evaluated and rejected (unknown structured code with refusal-prefixed prose, or an envelope over 16 KiB). Fix: write authorityRefusal: null as the explicit recorder verdict and legacy-decode only when the key is absent.

Low / non-blocking: cause is a dead dimension (REFUSAL_CAUSES empty, authorityErrorMeta never emits it); the redaction-version bump collapses legacy evidence pointers and causes a one-time signature discontinuity for UUID-bearing symptoms — both worth a line in the changeset.

Full write-up with reproduction steps is kept in the project store (docs/pr-1034-fable-review.md).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c66ee38743

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

assert.ok(calls.includes('stale'));
});

test('GH#202 runFlowParked: parking trace records begin, released, and committed', async () => {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Move the added parking diagnostics tests to TypeScript

These three new test cases expand a grandfathered .js test file, even though repository policy requires all new test code to use .ts or .tsx. Move the added coverage into a TypeScript test rather than growing the JavaScript baseline.

AGENTS.md reference: AGENTS.md:L45-L47

Useful? React with 👍 / 👎.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e4a2b29e09

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[/~\/[A-Za-z0-9_./-]+/g, '[PATH_REDACTED]'],
[/\/(Users|home|opt|var|tmp|etc|private|Volumes)\/[A-Za-z0-9_./-]+/g, '[PATH_REDACTED]'],
[/(com|org|io|dev|net)\.[a-zA-Z0-9_-]+\.[a-zA-Z0-9_.-]+/g, '[BUNDLE_REDACTED]'],
[/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi, '[ID_REDACTED]'],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Mirror UUID redaction into the feedback collector

When a simulator UDID appears in the CDP bridge log or legacy telemetry, this new rule protects only in-process evidence: scripts/collect-feedback.sh still passes those sources through a redact sed program with no UUID rule before including them in submitted feedback. This leaves identifiers such as 12345678-1234-1234-1234-123456789ABC intact despite the nearby contract requiring these sanitizer lists to stay aligned; add the equivalent rule to the root collector and regenerate both packaged host copies.

AGENTS.md reference: AGENTS.md:L314-L323

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Landed on this head. Root scripts/collect-feedback.sh and both packaged host copies now apply the same UUID → [ID_REDACTED] rule as the in-process sanitizer (b2ac1254 / 8ae939d).

Lykhoyda and others added 10 commits September 15, 2026 19:48
Take current main host bundles through the rebase, then rebuild so
authority-refusal reporting ships in both plugin copies.

Co-authored-by: Anton Lykhoyda <lykhoyda@gmail.com>
A recognized authority refusal persists the producer's error text after
sanitizeString, and METRO_ORIGIN_MISMATCH names the bound device by its
simulator UDID. The shared redaction rules did not cover UUIDs, so the
UDID landed in patterns.jsonl.

Add one UUID rule to REDACTION_RULES and bump REDACTION_RULES_VERSION so
retained rows re-sanitize on their next rewrite. Evidence pointers keep a
random id but drop the dashes so the rule does not collapse them.
Regenerate both host runtimes.
…traces

Axis is a reported attribute, not identity, so one code+platform stays one group. New failures persist an explicit non-membership verdict, and code-less action timeouts keep their runner traces.
The three parking-trace cases added in c66ee38 grew the grandfathered
gh-202-maestro-flow-parks-l2.test.js. New test code must be TypeScript,
so they now live in runner-diagnostics.test.ts beside the other
withRunnerDiagnosticsContext coverage, and the .js file is restored to
its main bytes. Call-order assertions already pinned by the existing JS
cases are not repeated.

Co-authored-by: Anton Lykhoyda <lykhoyda@gmail.com>
The in-process evidence rules gained a UUID rule in d4d7de4, but
scripts/collect-feedback.sh still sanitized CDP bridge log and legacy
telemetry tails with no such rule, so simulator UDIDs survived into
submitted feedback. Add the equivalent sed rule to the root collector,
regenerate both packaged host copies, and pin the collector path in the
existing UDID redaction test.

Co-authored-by: Anton Lykhoyda <lykhoyda@gmail.com>
Co-authored-by: Anton Lykhoyda <lykhoyda@gmail.com>
@Lykhoyda
Lykhoyda force-pushed the implement-react-native-development-agent-features branch from 2262b6f to 8ae939d Compare September 15, 2026 18:06
@Lykhoyda Lykhoyda changed the title fix(core): surface systemic authority refusal recurrence fix(core): report systemic authority refusals and retain runner timeout traces Sep 15, 2026
@Lykhoyda Lykhoyda added the needs-qa Independent dogfood QA on test-app simulator; Orchestrator posts a PR comment report label Sep 15, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HANDOFF: needs-qa

QA requested (needs-qa).
Linux automation cannot run device QA. GetSafe Project will spawn a new local Mac session.

QA_PR: #1034
PLATFORM: all
IMAGE_WIDTH: 800
POST_COMMENT: true

Claim: Report recognized authority refusals (SESSION_AUTHORITY_REQUIRED, METRO_ORIGIN_MISMATCH, RUNNER_OWNERSHIP_MISMATCH, HANDOFF_NOT_AUTHORIZED, NON_GIT_MANIFEST_REQUIRED, BUNDLE_HANDSHAKE_UNAVAILABLE) as one code-and-platform systemic group across gate, Error, failResult, and legacy-prose emission shapes (axis is reported but omitted from the grouping key; conflicting or missing axis becomes null). Persist an explicit authorityRefusal: null negative verdict on new and matching historical-row upserts so prose cannot re-admit non-membership, without backfilling untouched rows. Recognize code-less cdp_run_action TIMEOUT and producer timedOut and add observation-only flow-park / flow-stage diagnostics around existing park/stage/relaunch/cleanup/origin boundaries; do not change errors, cleanup, authority, or timeouts. The qa-observe-screen-match.yaml fixture is intentionally untouched. Prior note on this pair: Android QA FAILED; iOS already passed.

Changed paths:

  • packages/rn-dev-agent-core/src/experience/authority-refusal.ts
  • packages/rn-dev-agent-core/src/experience/evidence.ts
  • packages/rn-dev-agent-core/src/experience/trends.ts
  • packages/rn-dev-agent-core/src/experience/runner-diagnostics.ts
  • packages/rn-dev-agent-core/src/experience-trends.ts
  • packages/rn-dev-agent-core/src/tools/maestro-run.ts
  • packages/rn-dev-agent-core/test/unit/experience-authority-refusal.test.ts
  • packages/rn-dev-agent-core/test/unit/experience-systemic-trends.test.ts
  • packages/rn-dev-agent-core/test/unit/runner-diagnostics.test.ts
  • seed-experience/common-failures.yaml
  • scripts/collect-feedback.sh (and host copies)
  • .changeset/systemic-authority-refusals.md
  • Host runtime bundles under packages/{claude,codex}-plugin/rn-dev-agent-core/dist/

Local runner must:

  • New session (do not reuse an old chat; plugin ≥ 1.0.8 with /qa-pr)
  • Version preflight vs GitHub main + latest release before any device work
  • Own disposable git worktree (never the primary checkout)
  • Own dedicated simulator/emulator named with the PR identity, e.g. QA PR 1017 short-title
  • Plugin-repo PRs bind workspace test-app/
  • Report back on this PR with gh --attach and
  • Overall PASS or FAIL (PR/fix). Never PARTIAL.

Follow: https://github.com/Lykhoyda/rn-dev-agent-workspace/blob/main/.cursor/skills/cursor-project-qa/SKILL.md

Open in Web View Automation 

Sent by Cursor Automation: New bug for QA

@Lykhoyda

Lykhoyda commented Sep 15, 2026

Copy link
Copy Markdown
Owner Author

FAILED — Android FAILED · iOS NOT RUN · overall FAILED at exact head 8ae939d0eca1262239f2c4075e8fc07325b75d1a. Independent product acceptance of #1034 (issue #981), observed 2026-09-15 on a local Mac. Evidence for the captain; no approval, review verdict, branch edit, or merge decision.

What this head is for: the reporting corrections (A1/A2) and the B1 timeout traces are published here; the conditional stopApp:false fixture trial (B2) was not applied. This run is the launch / pin / replay gate — not the feature-proof chain. Green checks (25 successful, one neutral CodeQL) are not acceptance.

Identities

Item Value
Candidate 8ae939d0eca1262239f2c4075e8fc07325b75d1a, packaged core 1.0.8, 81 public tools
Workspace 7a63707af91b8bec20cab76f902a1553985251e6 — deliberately pinned; live main was one docs-only commit ahead (#59)
Saved action qa-observe-screen-match.yaml, unchanged before and after (blob 51a8dd42…)
Runner maestro-runner@1.1.24 from pin cache, payload verification passed
Android target one exclusive fresh emulator, API 36, headless, booted at load 5.1 with no pre-existing emulator

Android — FAILED

exclusive fresh emulator → URL bound before build → one managed build (signed envelope, exit 0)
  → attach-only open → one matching Hermes target → pin succeeds (56.8 s)
  → hideDevMenu: no_menu_present → native Welcome, app-root, Skip / Next  ← picker-free usable baseline
  → unchanged action: TIMEOUT, zero completed steps → no visible result → repeat not run
  → B1 trace exported → reverse cleanup proven

Replay call: qa-observe-screen-match, autoRepair:false, forceReload:false, proofReplay:true, timeoutMs:90000. Public result: failureKind: TIMEOUT, underlyingFailure: "Maestro flow timed out", firstAttemptMs: 130334, empty first-attempt output. Nothing was tapped: no picker, tutorial, or rescue deep link.

B1 timeout trace — retained

collect-feedback.sh scoped to the exact session returned failureCode: TIMEOUT with 13 events, truncated: false:

Trace event Monotonic ms
flow-park begin / released / committed 1526 / 1707 / 1727
stage 0 execute-begin 1728
runner payload verification passed (pin-cache 1.1.24) 2349
stage 0 execute-complete 11484
stage 0 relaunch-begin, stopApp=true 11484
stage 1 execute-begin 131555
stage 1 cleanup-begin 131561
tool-outcome FAIL 134082

There is no relaunch-complete, no stage 1 execute-complete, and no cleanup-complete. The gap after relaunch-begin is 120,072 ms. At this head maestro-run.ts:461 records relaunch entry, :466 awaits the managed relaunch, :467 records its return; the catch can retain an origin error and continue to the next stage (:473).

Inference, bounded: the long wait sits inside the managed relaunch boundary after native stage 0 returned — not before runner parking and not in initial runner execution. This does not prove the internal cause, and it does not prove that changing stopApp would fix it. That is the diagnostic owner's next question.

Confound recorded honestly: host load rose above 300 during the Android journey (samples 356.6, 332.7) despite the boot gate being satisfied. That is a material performance confound; it does not turn a failed acceptance into a pass, and it does not erase the trace.

iOS — NOT RUN

The owned simulator was allocated but never booted. After Android cleanup the one-minute host load stayed above 280 (samples 346.4, 365.1, 354.8, 282.6), repeatedly failing the mandatory below-10 boot gate, and the run was finalized rather than booting into a starved host. No iOS build, baseline, replay, or repeat exists on this head. Missing platform coverage is FAILED overall; nothing about iOS was tested, so it is recorded as not run rather than as a product failure.

Cleanup and an external inventory change

Owned cleanup is proven: runner close → stop_metrorestore_integrationrelease; postflight pass, cleanupProven: true; the unused iOS session cleaned in order; rnva cleanup reported simulator, Metro listener and integration residue all absent; zero emulator processes; workspace diff empty; action blob unchanged.

rnva cleanup nevertheless returned DEVICE_INVENTORY_CHANGED: between its pre- and post-run inventories, both initially booted foreign simulators were shut down, one new foreign QA simulator was booted, and one foreign AVD definition was added. This run issued no control against any foreign target, and the actor is unknown from these observations — it is reported as an external change, not claimed as unchanged.

Disposition

Return the B1 trace to the implementation owner: the managed relaunch interval is now directly bounded by diagnostic events. Investigate that boundary under controlled host conditions before choosing a behavioral fix. The fresh iOS journey remains owed once the host is idle. #1037 is out of scope. No merge.

Android after pin: the Test App's own Welcome screen with app-root, Skip and Next — no picker, no tutorial

@Lykhoyda

Lykhoyda commented Sep 15, 2026

Copy link
Copy Markdown
Owner Author

PR: #1034
Head: 8ae939d
Verdict: FAIL

Additive iOS-only coverage from a second independent local Mac session at the same head. It fills the iOS NOT RUN gap in #1034 (comment); it does not re-run or contradict that run's Android TIMEOUT. Overall stays FAIL because Android failed there. This is initial needs-qa evidence, not a merge gate. No approval, branch edit, or merge.

Target Result Evidence
ios PASS dedicated sim; pin_dev_clientcdp.connected: true; picker-free Welcome baseline; unchanged qa-observe-screen-match replay passed: true (transport cdp-js, proofDomain react-tree), route QaAcceptanceResult; cdp_error_log 0; screenshots + video below
android NOT RUN this session bound a dedicated emulator and started the managed build, then stopped before install/pin once the other run's report landed; no Android journey evidence here — the other comment's Android FAIL stands
device SKIP no exclusive USB serial

Plugin under test: isolated worktree at 8ae939d0 (committed host bundle packages/claude-plugin/rn-dev-agent-core/dist/supervisor.js, serverInfo.version 1.0.8, 81 tools). Workspace test-app (com.rndevagent.testapp) in its own worktree at workspace main 3fcee51. Installed Cursor plugin 1.0.8 equals GitHub main packages/claude-plugin/package.json 1.0.8 (cache hash = main HEAD 8601c963); release v1.0.9 is tagged on the unmerged Version Packages PR #1019, so no newer installable plugin exists. Cdp MCP discovery was error; session driven by supervisor fallback with cwd = that test-app worktree and a private RN_DEV_AGENT_EXPERIENCE_DIR. gh 2.100.0.

iOS journey

bind_device (dedicated sim QA PR 1034 authority refusals) → preview_integrationapply_integration confirmed=true → managed pnpm run ios from the app root → metroBound/installBoundpin_dev_client (bundleBound: true) → cdp_status cdp.connected: true, installIdentity: verifieddevice_snapshot open attachOnlydevice_record startcdp_run_action qa-observe-screen-match autoRepair=falsepassed: true, 27 s, per-step readback complete → cdp_navigation_state QaAcceptanceResult (stack Tabs > QaAcceptanceStart > QaAcceptanceResult) → device_record stopcdp_error_log 0 errors.

Reporting claim, observed on the private store (real product refusals only; nothing injected)

  • Codes emitted live by the candidate through public tools: BUNDLE_HANDSHAKE_UNAVAILABLE (cdp_component_tree, cdp_navigation_state; pre-pin and after a public device_reset_state relaunch=false terminate), RUNNER_OWNERSHIP_MISMATCH (device_screenshot before the runner was opened, axis R), HANDOFF_NOT_AUTHORIZED (accept_handoff with an unknown id; no meta.axis), SESSION_AUTHORITY_REQUIRED (three distinct prose messages, axis C: restore while Metro/runner active, release before restore, restore with no binding).
  • Shipped experience-trends grouped them one group per code and platform across tools: BUNDLE_HANDSHAKE_UNAVAILABLE | axis: B | platform: unknown | 4 occurrence(s) | tools: cdp_component_tree, cdp_navigation_state; SESSION_AUTHORITY_REQUIRED split only by platform (unknown vs ios), as designed.
  • Every stored systemicKey recomputes as sha256(["rn-dev-agent/authority-refusal/2", code, cause, platform]) — axis is reported but not in the key (A1).
  • Non-member failures (APP_INSTALL_IDENTITY_CHANGED, SCREENSHOT_FAILED, OPERATION_ALREADY_IN_PROGRESS) persisted an explicit authorityRefusal: null, including on repeat upserts (counts 2 and 7) (A2 on new rows).
  • recoveryCount stayed 0 on every recognized refusal despite successful status/inventory calls between them; report text says recovery not verified, authority state unknown.
  • A typed RUNNER_OWNERSHIP_MISMATCH retained a runner-diagnostics bundle (2 events, truncated: false).

NOT RUN here: B1 code-less TIMEOUT flow-park/flow-stage trace (no timeout occurred on iOS; covered by the other run on Android), METRO_ORIGIN_MISMATCH (pin_dev_client relaunched cleanly after the public terminate), NON_GIT_MANIFEST_REQUIRED, mixed-emission same-code collapse (gate axis vs no-meta), legacy keyless-row upsert (fresh store), 16 KiB boundary. Structured-code precedence and boundaries remain unit-verified only.

Side note (not a verdict input): interrupting the managed rn-session-adapter mid-build left the session fenced (OPERATION_ALREADY_IN_PROGRESS on every rn_session action with no Metro listener); a supervisor restart minted a new session with no integration binding, so restore_integration refused and the disposable worktree was reset by hand. Possible stuck-fence recovery gap, separate from this PR.

Repro steps (iOS)

  1. Pin plugin worktree at 8ae939d0eca1262239f2c4075e8fc07325b75d1a; supervisor cwd = isolated workspace test-app.
  2. Create a dedicated iOS simulator; rn_session bind_device with that exact UDID + com.rndevagent.testappapply_integration confirmed=truepnpm run ios → poll until metroBound/installBoundpin_dev_client.
  3. cdp_status → expect cdp.connected: true. cdp_run_action actionId=qa-observe-screen-match autoRepair=false → expect passed: true, route QaAcceptanceResult, cdp_error_log 0.
  4. Run rn-experience-trends --json against the session's RN_DEV_AGENT_EXPERIENCE_DIR; assert one systemicRefusals row per code+platform, recoveryEvidence: not-verified, explicit authorityRefusal: null on non-member rows.

Expected: iOS journey passes end to end; refusals aggregate per code+platform.
Actual: as expected on iOS.

Cleanup: runner close → stop_metrorestore_integrationrelease (iOS, in order); Android session released after the fence was cleared; own simulator and AVD deleted; both disposable worktrees removed; primary checkouts untouched; foreign devices not controlled.

ios

iOS after pin_dev_client: test-app Welcome onboarding, no Expo sheet iOS after strict qa-observe-screen-match replay: QaAcceptanceResult screen
ios-flow.mp4

@Lykhoyda

Lykhoyda commented Sep 15, 2026

Copy link
Copy Markdown
Owner Author

Quiet-host Android observation at 8ae939d0

#1034

Outcome: quiet-host failure. The unchanged action timed out with zero completed authored steps. One-minute load never exceeded 50 (peak 25.91). This is a valid quiet-host comparison, not confounded-by-load.

This is a diagnostic observation, not dual-platform acceptance. Published A1/A2/B1 stay. launchApp.stopApp remains true. No YAML/fixture/product edit. No merge.

Picker-free Welcome after pin

What ran

Workspace pin 7a63707af91b8bec20cab76f902a1553985251e6 (detached; live main was a docs-only commit). Action test-app/.rn-agent/actions/qa-observe-screen-match.yaml blob 51a8dd42dd7009e3275af7383114bda2edae77fe unchanged before and after. Candidate core 1.0.8, 81 tools, maestro-runner@1.1.24 pin-cache. One exclusive owned Android emulator. Pixel_9_Pro was not used.

Picker-free Welcome after pin: app-root, Skip, Next. cdp_dev_settings(action="hideDevMenu") returned executed:false, outcome:no_menu_present, surface:app. No picker, tutorial, or rescue deeplink was tapped.

Replay: cdp_run_action actionId=qa-observe-screen-match platform=android autoRepair=false forceReload=false proofReplay=true timeoutMs=90000.

Public result

{"ok":false,"error":"cdp_run_action: qa-observe-screen-match failed (TIMEOUT) — auto-repair disabled: Maestro flow timed out","meta":{"failureKind":"TIMEOUT","underlyingFailure":"Maestro flow timed out","firstAttemptMs":128906,"firstAttemptOutput":"","terminal":{"completedSteps":0,"exitClass":"timed-out"}}}

No relaunch exception or last-connect error is in the public envelope. Duration alone is not treated as a named deadline leaf.

B1 trace (13 events, truncated:false, failureCode: TIMEOUT)

Event Monotonic ms
flow-park committed 1,678.843
stage 0 execute-begin 1,679.111
payload-verify passed (1.1.24 pin-cache) 2,308.552
stage 0 execute-complete 10,115.943
stage 0 relaunch-begin (stopApp:true) 10,115.985
stage 1 execute-begin 130,178.946
stage 1 cleanup-begin 130,180.260
tool-outcome FAIL 130,942.023

No relaunch-complete. Gap after relaunch-begin: 120,062.961 ms. Native stage 0 returned; authored steps did not.

Load (1-minute)

Phase 1-min
Gate / boot 4.52
After pin 18.32
Action start 12.79
During relaunch interval 12.17–19.48
Timeout 15.85
Series max (around build) 25.91

QA4's load >300 confound does not apply here. The same stall reproduced on a quiet host.

Cleanup

Owned emulator, private AVD, unused validator simulator, Metro, and package integration were removed. Foreign shared AVDs Pixel_9_Pro and Pixel_10a were observed present and unused. No USB phone. Action bytes unchanged.

iOS remains NOT RUN for this head. Product acceptance remains FAILED. No merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-qa Independent dogfood QA on test-app simulator; Orchestrator posts a PR comment report

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants